Papers with LoRA fine-tuning
CrowdSelect: SyntheticInstruction Data Selection with Multi-LLM Wisdom (2026.findings-eacl)
Copied to clipboard
| Challenge: | Existing methods for capturing instruction-following complexity rely on single-dimensional signals, but they fail to capture complexity across diverse fields. |
| Approach: | They propose three foundational metrics that leverage Multi-LLMs wisdom to capture instruction-response pair characteristics and propose CrowdSelect, an integrated metric incorporating a clustering-based approach to maintain response diversity. |
| Outcome: | The proposed metrics outperform existing models on MT-bench and Arena-hard and show improvements of 4.81% on full and LoRA fine-tuning. |
Forgetting before Learning: Utilizing Parametric Arithmetic for Knowledge Updating in Large Language Models (2024.acl-long)
Copied to clipboard
| Challenge: | Existing methods of model editing and knowledge updating add additional network parameters, knowledge bases, knowledge base, and model parameters. |
| Approach: | They propose a new paradigm for fine-tuning called F-Learning that employs parametric arithmetic to facilitate the forgetting of old knowledge and learning of new knowledge. |
| Outcome: | The proposed model outperforms existing models on two datasets and is comparable to full fine-tuning and LoRA fine-uning. |
Can Large Language Models Act as Ensembler for Multi-GNNs? (2025.emnlp-main)
Copied to clipboard
| Challenge: | Existing graph neural networks lack the inherent semantic understanding capability of rich textual attributes, limiting their effectiveness in applications. |
| Approach: | They propose a model that integrates multiple GNNs and LLMs to provide an ensemble for multi-GNNs. |
| Outcome: | The proposed model outperforms existing models in terms of semantic understanding of graph structures and graph structures. |